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young Latino English learners with 45 teachers and 105 students in 12 ele- 
mentary schools. School-based teams randomly assigned to the intervention 
received professional development focused on cultural wealth, high-impact 
instructional strategies, and a framework for collaboration. We observed 
each teacher three times during the school year and assessed students indi- 
vidually at the beginning and end of the school year using the Woodcock 
Munoz Language Survey (WMLS). Using an intent-to-treat (ITT) analysis, 
we found effects for the intervention on teachers’ implementation of high- 
impact instructional strategies and students’ language and literacy skills. 


Keyworps: professional development, English learners, collaboration, 
literacy 


here are currently over 5 million English learners (ELs) in U.S. schools 

(National Center for Education Statistics [NCES], 2015), and over 70% 
speak Spanish as their first language (Migration Policy Institute, 2015). The 
majority of classroom teachers, however, have not had specialized training 
in working with ELs. In fact, fewer than 20% of teacher education programs 
require a course focused on ELs (U.S. Government Accountability Office 
[GAO], 2009). Latino ELs, on average, experience a persistent achievement 
gap in math and reading (Garcia, Jensen, & Scribner, 2009) and are at higher 
risk for dropping out of school (Bohon, Macpherson, & Atiles, 2005). School 
district administrators have indicated a need for teacher professional devel- 
opment on understanding cultural issues, assessing student progress, and 
developing instructional strategies for ELs (U.S. GAO, 2009). Clearly, there 
is an urgent need for teacher professional development (PD) that focuses 
on best practices for working with English learners. 

English learners are often working toward two milestones at once: English 
language mastery and acquisition of academic content knowledge (Calder6n, 
2007; Cloud, 2002). To reach these milestones within the present teacher prep- 
aration context, the current state of instructional affairs for ELs is untenable. Yet 
teachers cannot be expected to implement what they have not been taught. 
Some researchers have suggested that there is an “implementation gap,” refer- 
ring to the lack of congruency between evidence-based practices that work and 
actual instructional practice in schools (Garcia et al., 2009). Many regular class- 
room teachers will have at least one EL in their classroom, yet only 29% of 
teachers with ELs have had any professional development focused on enhanc- 
ing their instruction for ELs (Ballantyne, Sanderman, & Levy, 2008). 

In many schools, English as a second language (ESL) teachers provide direct 
instruction in the English language to students in a traditional pull-out model 
while ELs spend the majority of their day in the regular classroom (U.S. GAO, 
2009), thus often separating the instruction in language development from con- 
tent instruction. A recent policy brief by Hakuta and Pecheone (2016) stresses 
the importance of policies that provide instruction integrating English acquisition 
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Figure 1. \llustration of the Developing Collaboration and Consultation Skills pro- 
fessional development components. 


and academic content. Although ESL and classroom teachers work together in 
co-teaching models in some schools (Dove & Honigsfeld, 2010), most teachers 
have not had preparation in how to work collaboratively with colleagues 
(Goddard, Goddard, & Tschannen-Moran, 2007; Vangrieken, Dochy, Raes, & 
Kyndt, 2015). Therefore, in addition to professional development on high- 
impact instructional strategies to support ELs’ language and content learning, 
teachers also need a framework for effective collaboration to close the “imple- 
mentation gap” (Garcia et al., 2009). 

The purpose of the current study was to conduct an initial investigation 
of a professional development intervention program called Developing 
Collaboration and Consultation Skills (DCCS). The professional develop- 
ment program was designed to address the implementation gap (Garcia 
et al., 2009) and support classroom and ESL teachers’ collaboration, use of 
high-impact instructional strategies for language and literacy, and incorpora- 
tion of students’ cultural wealth (Yosso, 2005) into the classroom. The PD 
program includes a five-day summer institute and four additional content 
modules. In addition, on-site instructional coaching was available approxi- 
mately every six weeks from a member of the research team. Each school- 
based team (ESL and classroom teachers) collaborated for 30 minutes 
weekly to align both the content and instructional strategies for their ELs. 
See Figure 1 for an illustration of these PD components. 


Theoretical Framework and Related Research 


In considering the multiple theories that inform this study, research and 
theory cluster around three related areas—professional development, 
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collaboration and coaching, and literacy teaching and learning. Each of these 
areas is discussed further in the following. 


Professional Development 


The intervention is based on the research on characteristics of effective 
professional development programs. Considerable research has been con- 
ducted on the efficacy of various forms of PD (Garet, Porter, Desimone, 
Birman, & Yoon, 2001). However, an exhaustive review of over 1,300 studies 
by Yoon (2007) found that only 9 met the requirements of credible evidence 
as defined by the What Works Clearinghouse. Yoon and colleagues identi- 
fied six key characteristics of PD programs across the 9 well-designed inves- 
tigations: (a) workshops, (b) outside experts, (c) ongoing delivery, (d) 
follow-up support, (e) activities in context, and (f) content. Learning a com- 
plex task, unsurprisingly, takes committed, engaged time (Garet et al., 2001), 
and Yoon and colleagues found the threshold to be a minimum of 14 hours. 

Guskey (2002) highlights three aspects of teacher PD programs: (a) 
Change is gradual, (b) it is a process, and (c) it can be challenging. Guskey 
also suggests the development process is likely more cyclical than strictly lin- 
ear and occurs with incremental changes in teachers’ efficacy. A program’s 
structure and process must include a mechanism (e.g., coaching and collabo- 
ration) to support teachers so they can successfully engage in the challenges 
presented throughout the change process and consider how the PD may offer 
a positive alternative to current practice. Professionals must be provided with 
a supportive experience that will foster their acquisition of the skills necessary 
to further develop their practice. Finally, teachers must be provided with con- 
tinued follow-up support and be accountable for implementation. In combi- 
nation, these three principles all highlight the need for teachers to receive 
competent, respectful, thoughtful, data-driven support from engaged col- 
leagues (e.g., a coach or collaborators). 

Two additional related frameworks informed the design of the profes- 
sional development: scaffolding (Vygotsky, 1978) and the gradual release 
of responsibility (Pearson & Gallagher, 1983). Specifically, the PD program 
was carefully designed to support teachers’ learning of the instructional strat- 
egies, the collaboration model (Babinski, Sanchez, Knotek, Amendum, & 
Corra, 2013), and approaches for incorporating families’ cultural wealth 
(Babinski, Sanchez, Amendum, & Knotek, 2016) by building on Yosso’s 
(2005) model that maps the various types of capital that Latino families bring 
to the classroom, including aspirational, familial, linguistic, navigational, 
resistant, and social capital. 

Much of the PD activity involves teachers’ active participation through 
role playing and/or working with student data based on findings from 
a review of teacher preparation programs (Risko et al., 2008), which con- 
cluded that intensive teacher training programs emphasizing “learning by 
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doing” can produce better teacher knowledge, beliefs, and practices in com- 
parison to programs that emphasize only knowledge and beliefs. 


Collaboration and Coaching Frameworks 


Collaboration is characterized as a group-based activity that is task- 
focused and utilizes effective communication, reflection, and critical thinking 
in the service of providing better service to students (Homan, 2004; Meirink, 
Imants, Meijer, & Verloop, 2010). Two overarching factors facilitate effective 
teacher collaboration—structural components and process characteristics 
(Vangrieken et al., 2015). Structural components include common planning 
time and the explicit description of roles and group norms. Process character- 
istics include relationship building, mutual respect, a task focus, and mutual 
leadership. The result of successful collaboration includes teachers’ use of 
more innovative pedagogies and students’ improved understanding and 
school performance (Egodawatte, McDougall, & Stoilescu, 2011). 


Literacy Teaching and Learning 


The PD program also focuses on key conceptual literacy domains: phone- 
mic awareness, phonics, fluency, vocabulary, text comprehension (NICHHD, 
2000), and writing because of the clear benefit for ELs (August & Shanahan, 
2006). However, August and Shanahan (2006) emphasize that the five key lit- 
eracy domains are necessary for ELs but not sufficient, stating that sensitive 
modulation of instruction and a focus on English language development are 
vital. In a review of reading instruction for ELs over the past 20 years, research- 
ers (Amendum & Fitzgerald, 2011) discussed implications for practice based 
on the review of findings, including those related to development and key 
conceptual literacy domains. They stated, “If teachers thoughtfully adjust 
effective practices for English-speaking students, such as modifying lesson 
pacing or overemphasizing certain facets of instruction, the same practices 
often used with English-only students can be effective for English-language 
learners’ reading achievement” (Amendum & Fitzgerald, 2011, p. 388). 

Across the key conceptual literacy domains, the PD program prioritizes 
particular domains over others. Specifically, prioritization is based on the 
concept of constrained versus unconstrained literacy skills made popular 
by Paris (2005). Constrained skills include those that students can master 
over a relatively short time, such as phonemic awareness and phonics. 
Unconstrained skills are those that continue to develop across time, 
such as vocabulary and comprehension. Given National Assessment of 
Educational Progress (NAEP) results (NCES, 2014a, 2014b) from 2004, 
2008, and 2012 (that show a persistent gap in reading comprehension 
achievement of 31, 30, and 34 points between ELs and non-ELs, respec- 
tively), it is vital that particular instructional attention is paid to teaching 
and learning unconstrained skills for ELs that focus on comprehension. 
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Table 1 
Instructional Strategies for Language and Literacy 
Type of Skill Instructional Strategy Purpose/Literacy Domain 
Constrained 
Say It, Move It Develop phonemic awareness, phonics 
knowledge, and segmenting 
Blend as You Go Develop phonemic awareness, phonics 
knowledge, and blending 
Word Card Integrate strategies for word recognition: 
meaning, blending, and syntax 
Unconstrained 
Teaching Cognates Leverage L1 to recognize words and 
understand meaning in L2 
Vocabulary Text Talk Understand the meanings of words, 
especially academic vocabulary 
Frayer Model Understand the meanings of concepts and 
words, especially in content areas 
Modified DRA Promote comprehension, vocabulary, and 
understanding 
Sentence Frames—Oral Build oral language to internalize academic 
vocabulary 


Sentence Frames—Written Support written language to internalize 
vocabulary and sentence structure 


Consequently, the PD program addresses both constrained and uncon- 
strained skills but provides much greater emphasis on unconstrained skills, 
such as vocabulary and comprehension. See Table 1 for a description of the 
instructional strategies aligned to key literacy domains. 


Components of the Professional Development Program 


The teacher professional development program provides ESL and class- 
room teachers with a framework for collaboration, skills for implementing 
high-impact instructional strategies, and approaches for incorporating fami- 
lies’ cultural wealth into the classroom. As described earlier, the components 
of the PD program include the Summer Institute, four follow-up application 
modules, in-school instructional coaching, and peer collaboration through- 
out the school year (Babinski, Amendum, Knotek, & Sanchez, 2017). 
Following Guskey’s (2002) model, the Summer Institute lays the ground- 
work for participants to begin to change their professional practice to sup- 
port the instructional needs of EL students. The PD program intentionally 
builds on the notions of scaffolding, the gradual release of responsibility, 
and learning by doing within the Summer Institute and the four application 
modules. For example, to scaffold teachers’ learning of the instructional 
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strategies, teachers progress from learning simpler word recognition strate- 
gies, to more complex strategies for teaching vocabulary and comprehen- 
sion, and finally to integrating multiple instructional strategies. 


PD Content Related to Instructional Strategies for Language and Literacy 


Much of the PD focuses around research-based instructional practices in 
language and literacy development for ELs embedded in families’ cultural 
wealth and discussed using a structured collaboration framework. Based 
on key resources (Amendum & Fitzgerald, 2011; August & Shanahan, 
2006; Echevarria, Vogt, & Short, 2010), the language and literacy content 
of the PD focuses on (a) instructional design and planning for delivery 
and (b) key content knowledge and instructional strategies. Each of these 
areas is explicated in the following. 


Instructional design and planning for delivery. The PD addresses three 
broad principles of literacy instructional design and planning for delivery for 
ELs: preparation, building English background, and strategies for learning 
(derived from Echevarria et al., 2010; Echevarria, Short, & Powers, 2006). 
Within preparation, teachers engage in setting both content and language 
objectives, considering key content concepts and potential adaptations 
and important supplementary materials. For example, in a given lesson, 
teachers often create two sets of objectives, one for content (what will be 
learned) and another for language (how the content will be learned). 
Providing objectives to students implants a framework for the lesson to fol- 
low and allows students to acquire skills, strategies, and content. At the same 
time, key supplementary materials, such as manipulatives, realia, photo- 
graphs, multimedia, and demonstrations, can be used to a high degree to 
develop both language and content understanding. 

In building English background, teachers engage with linking concepts 
to students’ existing knowledge and background experiences, making 
explicit links between past learning and new knowledge in English and 
emphasizing key English vocabulary. In working with key vocabulary, 
teachers and students must engage in repeated oral and written use of 
new vocabulary to develop deeper understandings of word meanings as 
well as a breadth of vocabulary knowledge. 

Finally, for strategies for learning, teachers discuss ways to engage stu- 
dents, with many opportunities to engage in academic strategies to promote 
concept learning. Teachers give students opportunities to use (a) cognitive 
strategies, related to individual learning tasks; (b) metacognitive strategies, 
related to thinking about individual learning tasks, particularly in relation 
to knowing when cognitive strategies break down; and (c) social/affective 
strategies in which learning is enhanced through working with others. 
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Key literacy domains and instructional strategies. The PD provides 
teachers with a framework (see Table 1) for supporting students’ comprehen- 
sion of text and oral language development during instruction using two related 
strategies: a modified directed reading activity and use of language frames to 
summarize the text. Based on research that links students’ English oral language 
development with English reading comprehension (e.g., Lesaux, Crosson, 
Kieffer, & Pierce, 2010), the program uses a modified version of the widely 
used Directed Reading Activity (Betts, 1946; Spiegel, 1981) designed to support 
ELs’ oral language development and reading comprehension. Teachers support 
students as they first develop relevant background knowledge and practice oral 
language structures related to the text. Then, ELs read the text with support for 
comprehension from the teacher. Following the reading, teachers use language 
frames (e.g., Donnelly & Roe, 2010) to support students’ comprehension 
through summarizing (or other comprehension-related skills, such as identify- 
ing main idea and supporting details), designed to enhance oral language 
development as well as comprehension. 

English vocabulary development is also of primary importance for ELs, 
and three key vocabulary strategies are included in the PD: direct instruction 
in word meanings (derived from Beck & McKeown, 2001), use of Spanish/ 
English cognates (Carlo et al., 2004), and representing word meanings with 
a graphic organizer to support students’ representations of a target word’s 
meaning, as well as examples and non-examples (Frayer, Frederick, & 
Klausmeier, 1969). 

Teachers also learn effective instructional strategies for phonemic 
awareness and phonics knowledge. Specifically, three multisensory instruc- 
tional strategies are used to develop phonemic awareness, phonics knowl- 
edge, and word recognition in context: Say It Move It, Blend as You Go, 
and Word Card. These strategies assimilate multiple early reading skills: 
They demonstrate the alphabetic principle, help students learn phoneme- 
grapheme (sound-symbol) relationships, develop students’ segmenting 
and blending abilities (phonemic awareness tasks) in the context of read 
words, and help students learn to recognize sight words. Say It Move It 
and Blend as You Go were adapted from word work strategies that 
are part of the Targeted Reading Intervention (TRI; Amendum, 2014; 
Amendum, Vernon-Feagans, & Ginsberg, 2011; Vernon-Feagans, Kainz, 
Hedrick, Ginsberg, & Amendum, 2013). The Word Card provides students 
with a visual reminder of a process that integrates word recognition strate- 
gies to read unknown words: re-reading, blending sounds, attempting the 
word, and checking for syntax and other contextual clues. 


Collaboration Framework 


The collaboration framework is introduced, modeled, and supported 
during the Summer Institute and is used over the course of the year in the 
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program. It consists of two critical components: (A) working alliance and (B) 
time, structure, and accountability (TSA). The working alliance focuses on 
the process characteristics of effective collaborations and establishes effec- 
tive communication, positive regard between the participants, and reciprocal 
leadership. The TSA critical component involves setting up the working rela- 
tionships within the structural component of the collaboration process. This 
TSA critical component comprises three subcomponents: time management, 
structuring the working relationships by outlining group norms and roles, 
and establishing accountability measures. The collaboration framework 
includes a focus on both language and content objectives, a plan for how 
an instructional strategy or strategies will be implemented in both the regular 
classroom and the ESL classroom, and how the teachers will know if the 
approach was successful. In the subsequent week’s collaboration meetings, 
the teachers are instructed to review the previous week’s plan, how it went, 
and any modifications they need to make going forward. The model allows 
for considerable flexibility in the teachers’ selection of the instructional strat- 
egy to focus on as well as how the strategy will be implemented. This 
emphasis on the teachers’ ownership of the approach is intended to promote 
the sustained use of the instructional approaches over time, even after the 
intervention supports from the developers has ended. 


Incorporation of Cultural Wealth Into the Classroom 


The concept of cultural wealth is a useful and important framework for 
working with Latino children and their families CYosso, 2005). Yosso’s 
(2005) model maps the various types of capital that Latino/a and other minor- 
ities have that might be overlooked or misread in contexts outside of their 
communities. Through learning about cultural wealth, teachers can begin to 
sustain students’ cultural wealth in the classroom, and teachers and schools 
that already work from a strengths-based perspective can engage in a common 
language about families. Teachers can use this model to understand and build 
on student strengths to extend the students’ funds of knowledge (Valdez & 
Lugg, 2010). 


Ongoing Support 


Teachers were provided with two types of ongoing support throughout 
the school year. First, instructional coaching was available at each school 
about every six weeks. A member of the research team who is a former 
ESL teacher and elementary school principal provided instructional coaching 
during one of the structured collaboration meetings between the ESL teacher 
and the classroom teachers at each school. Adapted from Knotek, Woods, 
and Enrico’s (2015) Implementation Coaching Framework, each coaching 
session included a discussion of the instructional strategies the teachers 
were implementing and a discussion about how they were aligning both 
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content and instructional approach across the ESL and the regular classroom 
settings. Second, the full group of teachers in the intervention group met for 
1.5 hours four additional times throughout the school year, as shown in 
Figure 1. Each of the modules included a discussion of selected instructional 
strategies with an emphasis on incorporating the targeted approach into the 
teachers’ regular routine. Teachers attended 35 hours of professional devel- 
opment during the Summer Institute and an additional 6 hours during the 
modules. 

Teachers also received ongoing support for enhancing their instruction 
for their ELs from their colleagues. During the weekly collaboration meet- 
ings, teachers discussed both content and language objectives and how 
they were going to align their approaches for working with the students. 
For example, one ESL teacher planned to teach vocabulary that the students 
would encounter in an upcoming science lesson in their regular classroom. 
Both the ESL and classroom teacher planned to use sentence frames to sup- 
port the ELs’ development of oral language using the new vocabulary words. 


The Current Study 


During the 2014-2015 school year, we conducted a pilot study of the 
DCCS Professional Development Program using a randomized controlled 
trial. We examined the following research questions: 


Research Question 1: Does participation in the professional development program 
increase ESL and classroom teachers’ use of high-impact instructional strategies 
as compared to teachers in the control group? 

Research Question 2: Do students in the intervention group show a greater 
increase in language and literacy scores as compared to the students in the 
control group? 

Research Question 3: Do students at different levels of English proficiency benefit 
more from their teachers’ participation in the intervention compared to stu- 
dents in the control condition? 


Method 


Randomization Procedures 


ESL teachers were recruited to participate in the study and invited two of 
their colleagues who taught kindergarten, first, or second grade. Each team 
of three submitted an application for the study, which included the princi- 
pal’s signature indicating that they understood that teachers in schools ran- 
domly assigned to the intervention group would need to collaborate for 
30 minutes each week. We matched schools on the percentage of students 
eligible for free or reduced-price lunch and then randomly assigned each 
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school within a pair to either the intervention group or a waitlist/control 
condition. 


Participants 
Teachers 


Forty-five teachers from 12 elementary schools within three school dis- 
tricts in the Southeast participated in the study. The schools were located in 
urban and suburban settings. The group included 15 ESL teachers and 30 
classroom teachers: 3 men and 42 women. On average, participants had 
taught for approximately 9 years (range, 1-29 years). One of the 30 class- 
room teachers held ESL certification in addition to elementary education 
licensure. All of the ESL teachers were ESL certified. Teacher demographic 
information is presented in Table 2. The ESL teachers provided ESL services 
in a pull-out model in which they worked with a small group of students 
within a separate ESL classroom. A few of the ESL teachers also occasionally 
provided push-in instruction in which they worked with a small group of 
students in their regular classroom. Of the 45 participating teachers, 1 
teacher moved to a different school midyear and left the study. 


Students 


Latino ELs in each of the participating classrooms who qualified for ESL 
services and spoke Spanish as their first language were invited to participate. 
Students who were bilingual but not eligible for ESL services were not 
included in the study. Bilingual research team members attended the open 
houses/back-to-school nights of the 12 participating schools to speak with 
parents of eligible students about the study. Consent forms were distributed 
to parents at the open houses and by teachers to parents who did not attend 
the open house. Of the 154 students who met the eligibility criteria, we 
obtained parental consent for 118 (76% of eligible students). Of these 118 
students, 72 (82% of eligible students) were in intervention classrooms, 
while 46 (67% of eligible students) were in the control classrooms. Fifty- 
nine percent of the participating students were male, 41% female. By the 
end of the school year, 10 students had transferred to other schools, and 1 
student refused to participate in study activities, for a 9% attrition rate. 
Two students had missing end-of-the-year data. The final data set included 
105 students with valid pre- and posttest data. 


Measures 
Teacher Implementation 


Two observation tools were used during classroom observations three 
times during the school year: the Classroom Quality for English Language 
Learners (CQELL) (Goldenberg, Coleman, Reese, Haertel, & Rodriguez- 
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Table 2 
Classroom and English as a Second Language (ESL) Teacher Demographics 
Variables Control Intervention Total 
Classroom teacher-level variables n= 16 n=14 n = 30 
Gender n (%) 
Male 2 (13) 0 () 27) 
Female 14 (88) 14 (100) 28 (93) 
Total years of teaching M (SD) 10.23 (7.79) 7.07 (4.92) 8.71 (6.64) 
Highest degree completed 1 (%) 
Bachelor 9 (56) 8 (57) 17 (57) 
Master’s 7 (44) 6 (43) 13 (43) 
Licensure 1 (%) 
PreK/K-6 grades 15 (94) 12 (86) 27 (90) 
7-12 grades 1 (©) 2 (14) 3 (10) 
ESL certification n (%) 
Yes 1 (6) 0 (0) 1() 
No 15 (94) 14 (100) 29 O7) 
SIOP (Sheltered Instruction Observation Protocol training 1 (%) 
Yes 11 (73) 6 (43) 17 (59) 
No 4 (27) 8 G67) 12 (41) 
Language proficiency n (%) 
Only English 10 (63) 11 (79) 21 (70) 
Beginner Spanish 4 (25) 214) 6 (20) 
Fluent Spanish 1 (©) 0 ©) 1 (3) 
Other language besides Spanish 1 (6) 1Q) 27) 
ESL teacher-level variables n=8 n=7 n=15 
Gender 1 (%) 
Male 0 (0) 1 (14) 1M 
Female 8 (100) 6 (86) 14 (93) 
Total years of teaching M (SD) 18.50 (7.23) 11.50 (6.10) 15.23 (7.42) 
Highest degree completed 1 (%) 
Bachelor 3 (38) 4 67) (47) 
Master’s 5 (63) 3 (43) 8 (53) 
Licensure 7 (%) 
PreK/K-6 grades 5 (63) 4 (57) 9 (60) 
7-12 grades 3 38) 3 (43) 6 (40) 
ESL certification n (%) 
Yes 8 (100) 7 (100) 15 (100) 
No 0 (0) 0 ©) 0 ©) 
SIOP training 1 (%) 
Yes 8 (100) 6 (86) 14 (93) 
No 0 () 1 (14) 1M) 
Language proficiency  (%) 
Only English 2 (25) 2 (29) 4 (27) 
Beginner Spanish 3 G8) 4 (57) 7 (47) 
Fluent Spanish 1 (3) 1 (14) 2 (13) 
Other language besides Spanish 2 (25) 0 (0) 213) 
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Mojica, 2012) and a researcher-created DCCS Observation Tool. The CQELL 
contains 88 items that measure key dimensions of the classroom that are 
grouped into 14 dimensions that make up two subscales: Generic Lesson 
Elements and English Learner Supports. Within the English Learner 
Supports subscale, we also examined the instructional dimension for 
Adapts Strategies. Members of the research team conducted all classroom 
observations. Online training (Goldenberg et al., 2012) is available for class- 
room observers and includes a series of instructional videos for practicing 
coding each of the instructional elements. In the pilot study, we obtained 
strong interrater reliability for CQELL for two trained observers, Cohen’s 
kappa of & = .82 (95% CI [.54, .65]), p < .001. 

We also observed teachers using the DCCS Observation Tool. The tool 
includes nine instructional strategies measured with 42 indicators. Videos of 
teachers implementing each of the strategies were used for training observers 
in using the tool. Although the DCCS Observation Tool was aligned to the 
DCCS instructional strategies, instructional indicators were coded for teachers 
in the control conditions who implemented similar instructional strategies. For 
each of the nine instructional strategies listed in Table 1, we developed a list of 
3 to 7 indicators necessary for accurate implementation of the strategy. The 
interrater reliability for the Observation Tool was in the moderate range 
with Cohen’s kappa of & = .79 (95% CI [.77, .87), p < .001. Because the 
DCCS Observation Tool is aligned to the DCCS PD, we included the CQELL 
to capture more general components of quality teaching for ELs. We hypoth- 
esized that the teachers in the control condition would be providing general 
support but may not be systematically enhancing their instruction or using 
specific strategies to support their ELs. 


Student Assessments 


Students were assessed using the Woodcock Munoz Language Survey- 
Revised Normative Update, (WMLS-R; Schrank, McGrew, & Dailey, 2010), 
which measures students’ language and literacy skills in both English and 
Spanish. The seven subscales include Picture Vocabulary (median reliability 
91), Verbal Analogies (median reliability .90), Letter-Word Identification 
(median reliability .97), Dictation (median reliability .94), Understanding 
Directions (median reliability .82), Story Recall (median reliability .76), and 
Passage Comprehension (median reliability .82). We assessed students at 
the beginning and end of the school year. The English WMLS-R has two 
forms; we administered Form A at the beginning of the school year and 
Form B at the end of the school year. The Spanish WMLS was administered 
at the beginning of the year and is used as a covariate in our analyses to con- 
sider and account for potential cross-linguistic transfer. For all subtests, W 
scores were calculated, which are Rasch ability scores providing equal inter- 
val characteristics of measurement. 


129 


Babinski et al. 


Table 3 
Student Characteristics 
Variables Control Intervention Total 
n 41 64 105 


Student-level variables 
Gender 1 (%) 


Male 30 (73) 34 (53) 64 (61)* 

Female 11 (27) 30 (47) 41 (39) 
Grade n (%) 

Kindergarten 13 (32) 24 (37) 37 (35) 

First grade 13 (32) 25 39) 38 (30) 

Second grade 15 (37) 15 (23) 30 (29) 
Age in years M (SD) 6.66 (.87) 6.36 (89) 6.48 (.89) 


Pretest Broad English Ability 438.68 (30.90) 430.50 (29.90) 437.35 (30.16) 
Total W score M (SD) 
Pretest Broad Spanish Ability 432.33 (19.25) 435.27 (18.33) 434.14 (18.05) 
Total W score M (SD)? 
District-reported variables 


Days absent M (SD) 8.13 (6.24) 8.33 (5.20) 8.25 (5.59) 
Free or reduced lunch n (%) 
Yes 31 (76) 56 (88) 87 (83) 
No 8 (20) 7QD 15 (14) 
Missing 2 (5) 1@Q) 3 @) 


471) = 4.22, p = .04. 
one 101; control 1 = 39; intervention 7 = 62. 


Demographics of the Teacher and Student Groups 


We examined the intervention and the control group teachers to deter- 
mine if there were any differences between the two groups in their gender, 
total number of years in teaching, degrees held, areas of teaching licensure, 
ESL certification, previous SIOP (Sheltered Instruction Observation Protocol; 
Echevarria et al., 2010) training, and languages spoken. No differences were 
found between the intervention and the control groups for any of these 
teacher demographic variables (see Table 2). We also examined the differen- 
ces between the students in the intervention group and the control group in 
their gender, grade level, age, number of days absent, eligibility for free or 
reduced-price lunch, pretest Broad English score on the WMLS, and pretest 
Broad Spanish score on the WMLS. Significantly more males were in the con- 
trol group (73%) as compared to the intervention group (53%). The students 
in the two groups did not differ on any of the other demographic variables 
(see Table 3). 
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Data Analysis 


All analyses other than descriptive statistics were conducted in Mplus 
v7.4 (Muthén & Muthén, 2015) using multilevel path-analytic analogs of 
ANOVA and ANCOVA to take advantage of more sophisticated analysis 
options in structural equation modeling software. We estimated effect sizes 
using Hedge’s g, which applies a sample size adjustment to Cohen’s d. 

We addressed the first research question about the teachers’ implemen- 
tation of the instructional strategies using observational data measured with 
the CQELL and the DCCS Observation Tool. Because the observational data 
tended to have low variability at individual occasions, we aggregated across 
the three observations to construct the outcome variables. The model used 
a binary coding of experimental condition as an intent-to-treat (TT) analysis 
predicting four instructional strategy measures. We selected the ITT analysis 
as a conservative approach to determine the impact of the intervention for 
both teachers and students. In addition, while our measure of implementa- 
tion fidelity documented the teachers’ use of the instructional strategies, it 
did not measure the quality of implementation. Because of the modest 
teacher-level sample size, these analyses did not incorporate covariates. 

The second research question assessed program outcomes using the W 
scores on the WMLS-R. We selected the W scores as they are designed as an 
interval-level continuous measurement of student competencies that have con- 
stant interpretation across ages and grades. As with teacher outcomes, we con- 
ducted an ITT analysis of condition predicting posttest scores on the seven 
subscales, covarying student gender, age, and start-of-year overall Spanish pro- 
ficiency as well as pre-intervention English proficiency on the seven subtests. 


Accommodating Missing, Non-Normal, and Multilevel Data 


For all analyses, we used maximum likelihood estimation from raw data 
with robust “sandwich” estimators of standard errors (‘MLR” in Mplus) to 
accommodate missing posttest data and non-normality. Multilevel models 
for teacher outcomes adjusted for clustering of teachers (Level 1) within 
schools (Level 2). Models for student outcomes adjusted for clustering of stu- 
dents (Level 1) within teachers (Level 2). The models involving only student 
data did not explicitly include Level 3. Because there were no teacher-level 
variables in those models, school-level variance components would be sub- 
sumed into teacher-level variance components. 


Results 
Research Question 1: Use of Instructional Strategies 


We examined the differences between teachers in the intervention 
group and the control group on three variables from the CQELL classroom 
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observation tool—Generic Lesson Elements, EL Support, and Adapted 
Strategies—as well as observer ratings of DCCS Strategies. Of the 15 ESL 
teachers in the study, 14 were observed providing instruction in the pull- 
out ESL classroom. One teacher was observed working with a small group 
of students in the regular classroom. For all teachers, the most frequently 
observed instructional strategies were Modified DRA and Vocabulary Text 
Talk. The least frequently observed strategies were the Frayer Model and 
the use of Word Cards. Ratings were aggregated across three observations 
as averages of non-missing values. No differences were found between 
teachers in the intervention group and the control group on CQELL 
Generic Lesson Elements or EL Support, indicating that the overall quality 
of instruction was similar between the two groups. However, the interven- 
tion did lead to greater use of Adapted Strategies (Maige = 0.57, 95% CI 
[.05, 1.10], p =.032, Hedge’s g = 0.95) and DCCS Strategies (Maire = 0.38, 
95% CI [0.27, 0.50], p < .001, Hedge’s g = 2.02) relative to teachers in the con- 
trol group. Table 4 shows the full results of these analyses. 


Research Question 2: Student Outcomes 


We examined the PD program’s effect on student outcomes using the 
seven subtests of the WMLS-R. We estimated a path-analysis analogue of 
a multivariate multiple linear regression model predicting the seven posttest 
W scores from experimental condition, covarying student gender, age, and 
start-of-year overall Spanish proficiency as well as pre-intervention English 
proficiency on the seven subtests. As is standard for multiple regression 
models, the path model was saturated and therefore fit the data perfectly. 

We found a significant, beneficial, ITT effect on Story Recall, with an esti- 
mated group difference in change in W scores of 3.02, 95% CI [0.02, 6.02], p = 
.048, Hedge’s g = 0.29. We also found an effect on Verbal Analogies; 
although the p value for the difference in change in Verbal Analogies W 
scores was not statistically significant (b = .053), the confidence interval 
was almost entirely above zero, Mai = 3.13, 95% CI [-0.12, 6.38], Hedge’s 
g = 0.23. Full results are shown in Table 5. 


Research Question 3: Exploratory Analysis of Differential Student Outcomes 


We also examined differences within groups of students at different lev- 
els of proficiency on the English subtests of the WMLS-R. We grouped stu- 
dents according to their pretest English proficiency on a given subtest: no 
proficiency (subtest score <.1 percentile), minimal proficiency (subtest 
score at or between .1 and 5th percentile), and some proficiency (>5th per- 
centile). Three of the WMLS subtests—Picture Vocabulary, Understanding 
Directions, and Story Recall—met the sample size criterion of at least five 
students in each condition for all three proficiency levels. Models for the 
remaining subtests included only the minimal proficiency and some 
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proficiency subgroups. These models were estimated in multiple-group path 
analyses with the same covariates as the full sample analyses previously. We 
conducted separate models for each of the seven subtests because the 
grouping was specific to a given subtest. 

We found that students in the intervention group with minimal pretest 
proficiency on the Story Recall subtest showed significantly greater progress 
than did students in the control group (adjusted M = 15.9 vs. 9.1; Maire = 7.8, 
95% CI [5.0, 10.7], p < .001, Hedge’s g = 0.24). Similarly, we found signifi- 
cantly greater growth for intervention students in the minimal proficiency 
group on the Letter-Word Identification (adjusted M = 90.1 vs. 58.7; Mairr = 
31.4, 95% CI [3.0, 59.8], p = .030, Hedge’s g = 0.58) and Dictation (adjusted 
M = 37.6 vs. 17.6; Maise = 20.0, 95% CI [1.5, 38.5], p = .034, Hedge’s g = 
0.34) subtests. On the Understanding Directions subtest, we found that inter- 
vention group students with no pretest proficiency made significantly 
greater gains than did students in the control group (adjusted M = 42.1 vs. 
30.6; Maire = 11.5, 95% CI [2.8, 20.3], p = .014, Hedge’s g = 0.20). We did 
not find a differential effect of the program for the students in the some pro- 
ficiency group (standard score above 75) on any of the subtests. Full results 
for all proficiency groups and subtests appear in Table 6. 


Discussion 


The purpose of this study was to examine the impact of a teacher pro- 
fessional development program on teaching practices and the language and 
literacy skills of young ELs. We found a positive impact of the PD program 
on teachers’ use of specific instructional strategies for ELs and students’ liter- 
acy outcomes. The teachers’ use of the instructional strategies suggests that 
teachers were able to implement the strategies with a high level of fidelity 
and that these strategies were substantially different from instructional 
approaches in the control classrooms. The overall quality of the classroom 
environment, as measured by the CQELL, was the same in both the interven- 
tion and the control groups, indicating that the key difference was the use of 
the DCCS instructional strategies. By using two observation scales, one that 
measures quality teaching for EL more generally and one that measures the 
implementation of our specific instructional strategies, we can begin to attri- 
bute the differences in student outcomes to the implementation of our 
approach rather than possible preexisting differences in instructional quality 
given the relatively small sample size. 

Previous research has highlighted the importance of measuring imple- 
mentation fidelity in studies that ask teachers to change their practices 
(Dane & Schneider, 1998). The DCCS Observation Tool provided an indica- 
tion of the extent to which teachers were implementing all steps for a given 
instructional strategy, a measure of program adherence (Dane & Schneider, 
1998). Future development of the DCCS Observation Tool should focus on 
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providing additional detail about the quality of the implementation to distin- 
guish among different levels of implementation among teachers. Our results 
suggest that it is important to use a fidelity measure that is closely aligned 
with the teaching strategies that one would expect to see in classrooms 
where the teacher has participated in the professional development program 
to identify the key differences between the new approach and standard 
teaching practices. Supporting teachers to implement high-impact instruc- 
tional strategies is the first step toward improving student outcomes. 

We also found a positive impact of the PD program on student out- 
comes. Students’ growth on two of the seven subtests of the WMLS indicated 
that students whose teachers participated in the PD program made greater 
gains as compared to students in the control classrooms. We found positive 
effects on the Story Recall subtest (Hedge’s g = 0.29), which measures oral 
language including listening skills, memory, and expressive language, and 
on the Verbal Analogies subtest (Hedge’s g = 0.23), which measures the abil- 
ity to reason using lexical knowledge. It is important to note that Hill, Bloom, 
Black, and Lipsey (2008) indicate that an effect size of 0.23 can serve as an 
empirical benchmark for intervention studies that examine student outcomes 
on a standardized measure with a narrow focus, such as the one used in this 
study, the WMLS. 

In taking a closer look at these two subtests, we noted that one explanation 
for these findings is that the PD program prioritized particular domains over 
others. Specifically, in the PD program, we discuss the concept of constrained 
versus unconstrained literacy skills (Paris, 2005). Constrained skills include 
those that can be learned to ceiling over a relatively short time, such as phone- 
mic awareness and phonics. Unconstrained skills are those that continue to 
develop across time, such as vocabulary and comprehension. One possible rea- 
son for the positive effect on Story Recall is the integration of multiple instruc- 
tional strategies by intervention teachers that focus on unconstrained skills. 
Intervention teachers specifically employed the Modified DRA strategy along 
with language frames to support students’ retelling and summarizing of texts. 
In addition, we observed teachers frequently using the Vocabulary Text Talk 
strategy. Repeated use of these strategies supported the ELs in acquiring the aca- 
demic language as well as the structures used in summarizing or retelling. The 
significant difference between intervention and control students on Story Recall 
may be an indication of students’ internalization of the key language structures 
and organization of an effective oral summary. 

Students from intervention classrooms outperformed those from control 
classrooms on the Verbal Analogies subtest, which measures reasoning using 
lexical knowledge, an example of another unconstrained skill. Intervention 
teachers likely spent significant amounts of time working with students on 
academic language structures through the use of language frames 
(Donnelly & Roe, 2010). Repeated use of language frames to internalize aca- 
demic language structures (e.g., academic language related to compare/ 
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contrast; [two things] are similar because ... , [two things] are different 
because . . .) with multiple texts may have contributed to the advantage 
on Verbal Analogies for students from intervention classrooms. 

In an exploratory analysis of how students at different levels of begin- 
ning English proficiency benefited from their teachers’ participation in the 
PD program, we found that students at lower levels of English proficiency 
were especially likely to benefit from their teachers’ participation in the 
PD program. Students whose standard score was between 55 and 75 at pre- 
test (below the 5th percentile and above the .1 percentile) showed signifi- 
cant improvement in three of the seven subtests of the WMLS. Similarly, 
students in the lowest proficiency group (below the .1 percentile; standard 
score below 55) were also more likely to benefit from the intervention, 
with significant improvement in one of the three subtests analyzed. 
Conversely, the progress of students above the 5th percentile (with a stan- 
dard score above 75) did not differ significantly from students in the control 
group. Students in this group were at or near expected English proficiency 
for children in their age group and may not have benefited as much from 
their teachers’ participation in the PD program. Furthermore, students in 
this higher proficiency group were less likely to receive direct instruction 
from the ESL teacher because they would be categorized as qualifying for 
“transitional” ESL services, which meant that the ESL teacher monitored 
the students’ progress and provided consultative support to the classroom 
teacher. In addition, classroom teachers may not have modified their instruc- 
tion for these students but rather focused on students with lower proficiency 
in English. Future studies should examine the students’ levels of proficiency 
and the teachers’ use of strategies that match their instructional levels. 


Limitations 


One limitation of this study is the relatively small number of schools and 
the necessity to randomize at the school level. Because many smaller ele- 
mentary schools have just one ESL teacher for K, first, and second grades, 
it was not possible to randomly assign teams of teachers to the intervention 
or control condition within schools. Additionally, a larger number of stu- 
dents in the study would be helpful for conducting subgroup analyses. A 
larger sample of students would allow for a more thorough examination 
of the impact of the PD program on students at various levels of proficiency. 

A second limitation of the study was the need to aggregate teacher 
observations across the school year due to limited variability. A more sensi- 
tive measure of fidelity that includes quality of implementation rather than 
simply adherence to the instructional strategies would allow for analyses 
of teacher improvement over the school year. In addition, a measure of 
the quality of implementation would allow for analyses of how different lev- 
els of implementation predicted student outcomes. 
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Implications for Policy and Practice 


A clear recommendation from this study is that with ongoing, school- 
based support for implementing innovations in the classroom, teachers 
can improve their instruction for English learners. Feedback from the teach- 
ers highlighted the importance of the ongoing PD modules and instructional 
coaching sessions at their schools. In addition, providing specific training in 
high-impact instructional strategies along with a framework for professional 
collaboration supports teachers in their implementation of new practices. 
This study also provides support for policy recommendations that integrate 
learning English with academic content, as proposed by Hakuta and 
Pecheone (2016). For example, ESL instruction is beneficial for pre-teaching 
academic vocabulary that the regular classroom teachers can build on to pro- 
mote comprehension in academic content areas. In addition, school policies 
that promote collaboration between ESL and classroom teachers have the 
potential to capitalize on the ESL teachers’ expertise as a way to support reg- 
ular classroom instruction for ELs. ESL and classroom teachers in this study 
appreciated the opportunity to engage in professional conversations about 
their specific teaching approaches and built productive collaborative rela- 
tionships to capitalize on the ESL teachers’ expertise in teaching language 
and the classroom teachers’ deep knowledge of the content areas and cur- 
riculum for students at their grade level. The next step for the teacher pro- 
fessional development program is a larger randomized controlled efficacy 
trial with a sufficient number of schools to analyze the data using a nested 
model to provide evidence of its effectiveness across many different types 
of school contexts. In addition, a more sensitive measure of implementation 
fidelity would be useful in determining what level of implementation pre- 
dicts student growth and which components of the PD model are essential 
to reap the benefits. 


Summary 


In summary, teachers in the PD program were more likely to implement 
high-impact instructional strategies for ELs as compared to teachers in the con- 
trol group. Importantly, the study also showed significant benefits to students 
in terms of significant growth in their English language and literacy skills as 
compared to students in the control classrooms. In particular, students in 
the intervention group at lower levels of English proficiency made signifi- 
cantly more progress as compared to students in the control group. 
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